A Comprehensive Resource to Evaluate Complex Open Domain Question Answering

We describe two corpora of question and answer pairs collected for complex, open-domain Question Answering (QA) to enable answer classification and re-ranking experiments. We deliver manually annotated answers to non-factoid questions from a QA system on both Web and TREC data. Moreover, we provide the same question/answer pairs in a rich data representation that includes syntactic parse trees and predicate argument structures and is compatible with the SVM-light toolkit. Experimenting with the above corpora allowed us to learn effective answer classifiers and re-rankers to improve the accuracy of our baseline QA system.